The invention concerns a process for the three-dimensional measurement of objects through optical exposures, projected patterns, and calculations of triangulation, in which the devices for the projection of the pattern and for the image capture are constructed separately from one another and can, for the measurement, be positioned independently of one another.
Furthermore, the invention concerns a device for such a process.
The use of processes that work on an optical basis offer a multiplicity of advantages for the purpose of measuring. A measurement can be carried out quickly and without contact. Electronic image converters, such as CCD arrays, for example, the output signals of which can be stored or evaluated after digitalization, already belong to the state of the art.
Processes and devices for the representation and three-dimensional optical measurement of spatial surfaces are already known. These are based upon triangulation processes in which patterns of points, lines, or any other chosen patterns are projected, at a certain angle, onto the surface under consideration, and the projected patterns are captured, under another viewing angle, by means of an optical device and an image converter. The known geometry between the direction of projection and the direction of capture permits the three-dimensional calculation of support points of the surface.
If a uniform pattern of lines is projected onto an object with spatial extensions, for example, then, because of the surface shape of the object, a distorted line pattern is brought about in a direction of view which is different from the direction of projection. If the object is represented from this direction of consideration on a CCD-array by means of a suitable optical device, and the image signals are digitalized and made available to a data processing unit, then it is possible to identify the lines at the various points of the image and, given a knowledge of the optical path of the rays, and under consideration to the geometrical configuration of the projected pattern of lines, to compute the 3D coordinates by means of triangulation.
In the capture of several individual images with different directions of capture, it is already state of the art to combine the partial information obtained about three-dimensional surfaces by means of the 3D coordinates of corresponding surface segments into a total information by means of so-called xe2x80x9cregistration-algorithmsxe2x80x9d.
It is now desirable to do the following for endoscopic investigations, for example:
a) To use a separate unit for the projection of the pattern and for the capture of the image, and to use this with an orientation that is freely selectable in relation to the surface to be measured. In order to obtain relevant data for the calculation of the triangulation, a minimum angle between the orientation of both of the units should not be fallen short of in this connection;
b) Insofar as a series of individual images is captured during the capture process, to change the position and orientation of both of the units independently of one another in relation to the object, such as for the simplification of the handling, for example, in order to be able to measure additional surface areas by means of additional image captures.
It is thus the task of the invention to create the process and the device for the three-dimensional measurement of objects of the type just described, in which the position and orientation of the image capture unit and of the projection unit can be adjusted to one another and, in the event that more than one capture is useful for this purpose, to follow the independent movement of the two units towards one another that has taken place during the course of the capture process. The geometrical connection between the image capture and the pattern projection determined for the specific individual image must then be taken into consideration for the calculation of triangulation.
This task is solved, in accordance with the invention, through the fact that:
a) Through the use of an electronic surface image converter, a capture process is carried out;
b) Through the use of a source of beams and suitable optical devices, a pattern is projected onto the object at an angle which is different from the direction of capture that is to be measured;
c) Through the use of a suitable unit, the output signals of the image converter are digitalized, and these data are made available to a data processing apparatus;
d) Through the use of a data processing apparatus, 3D coordinates of the points of the surface of the object to be measured are determined, by means of suitable circuits and/or algorithms, through calculations of triangulation from the image data of the individual images;
while additional devices are provided in such a manner, and/or the process coming for use and/or the devices are configured in such a manner, that the position and the orientation of the image capture unit and of the projection unit to one another can be determined. If the position and the orientation of both units to one another is known, then, for the capture of an individual image, 3D coordinates can be determined by means of a triangulation calculation, in the known manner and method, for a number of points of the surface.
It is, in accordance with the invention, advantageous to use additional devices for this purpose in order to measure the position and the orientation of the image capture unit and of the projection unit to one another.
In accordance with the invention, this can be carried out, in an advantageous manner, both directly as well as indirectly, both by means of scanning or in connection with measuring means, as well as without contact, such as by optical means, for example.
If information on the object which is additional to that information acquired from an individual exposure is now to be determined, then the image capture unit must be able to record a sequence of individual images. For the task in accordance with the invention, namely, that the projection unit and the image capture unit are moved differently during the course of the image capture process, the devices and/or processes in accordance with the invention must, in an advantageous manner, be configured in such a manner that the geometrical configuration of the image capture unit and of projection unit can be adjusted to one another for the sequential images.
This can, in accordance with the invention, be advantageously guaranteed through the fact that the above-stated measuring devices are also evaluated at the point in time of the exposures that are additionally carried out, and the measuring results are taken into consideration, in the way and manner described above, in the calculation of triangulation.
For the event that, in the sequence of the individual images, the object is recorded in such a manner that the 3D coordinates determined from the individual images describe surface segments that are in agreement to a considerable degree, thenxe2x80x94as has been described abovexe2x80x94the specific 3D information can be combined by means of xe2x80x9cregistration algorithmsxe2x80x9d corresponding to the state of the art.
If the capture of the individual images takes place in rapid sequence, then, with a corresponding handling of the units, it can be assumed thatxe2x80x94first of allxe2x80x94the portion of corresponding surface segments is sufficient andxe2x80x94secondlyxe2x80x94that the geometrical correspondence of the image capture unit and of the projection unit to one another has only been altered in extent in relation to the previous capture.
If, therefore, the coordination of both units in the first capture was known, for example, and if a considerable number of 3D coordinates describing the surface could be computed, then the information contained in the next capture differs in the following components:
a) The 3D coordinates of the corresponding surface segments can be represented displaced linearly in the three spatial directions;
b) The 3D coordinates of the corresponding surface segments can be represented displaced in a rotational manner in relation to the three spatial directions;
c) The relative position of the image capture unit and of the projection unit to one another can be changed linearly in the three spatial directions;
d) The relative position of the image capture unit and of the projection unit to one another can be changed in a rotational manner in relation to the three spatial directions.
For the purpose of the generalization of expressions, these possibilities of change will also be termed in the following as xe2x80x9cdegrees of freedomxe2x80x9d.
Matching algorithmsxe2x80x94which form, from the different data records of 3D coordinates, the intersections of that data which are to be assigned to corresponding surface segments, and which vary and optimize the possible changes (a) and (b) listed above in such a manner that the intersection of the specific 3D data of both data records fit optimally with one another in regard to the corresponding displacements and rotationsxe2x80x94already belong to the state of the art in the technical area of graphic data processing. One presupposition for this is that the changes (c) and (d) listed above are known.
Algorithms which systematically vary a finite number of degrees of freedom and optimize these in accordance with one or more criteria of quality are already known from numerical mathematics. In a systematic variation, however, the computational expense is raised to a higher power with every additional level of freedom.
Recursively gradated processes which, through a stochastic variation of the degree of freedom and a step-wise selection of each specifically best variant, under the assumption that the quantities of information that are to be evaluated are essentially similar, reach an optimal solution surprisingly quickly, are already known, in connection with the phrase xe2x80x9cevolution strategiesxe2x80x9d, from the field of bionics.
In this sense, it is, in accordance with the invention, advantageous for the solution of the task in accordance with the invention to use expanded matching algorithms in such a manner that:
First of allxe2x80x94In a recursive numerical process, multiple so-called xe2x80x9csuccessorxe2x80x9d variants are first formed through the stochastic variation of the above-stated degrees of freedom (c) and (d);
Secondlyxe2x80x94For every successor variant, the 3D data records are, in the known manner, computed from image information by means of triangulation, in accordance with the assumptions made for the degrees of freedom (c) and (d);
Thirdlyxe2x80x94For every successor variant, the intersections of those data which are to be assigned to the corresponding surface segments are formed, in the known way and manner, in a matching computation from the different data records of 3D coordinates, and the above-listed degrees of freedom (a) and (b) are varied and optimized in such a manner that the intersections formed from the specific 3D data of both data records can, under consideration of the specifically-selected geometrical change, be optimally transformed into one another in relation to the level of freedom (a) and (b);
Fourthlyxe2x80x94Out of all of the successor variants, that one which has the maximum value achieved in regard to the agreement of the data from both data records describing the same surface segment is selected in the sense of an evolution;
Fifthlyxe2x80x94Corresponding to the evolution strategy described, and proceeding from the assumptions that were made for the degrees of freedom (c) and (d) of the previously-selected best successor variants with reduced boundaries of the degrees of freedom (c) and (d), new successors of the next generation are again stochastically varied in different variants and, in a manner corresponding to the above-stated five pointsxe2x80x94first to fifthxe2x80x94it is then proceeded in a recursive manner for long enough until the boundaries of variation of the degrees of freedom (c) and (d), and the differences between the generations in regard to the maximum value achieved in every passage, are sufficiently small.
As the result, all information required is known with sufficient precision.
It is, in accordance with the invention, advantageous to construct the devices for the projection of the pattern and for the capture of the image in such a manner that these are connected rigidly and in an unambiguous position and orientation with one another during the capture of the first individual image. During the further course of the capture, the connection is detached and a movement independent of one another is thereby made possible. The additional three-dimensional measurement values of the object can then be computed, for every new image capture, with the expanded registration algorithm described above and, as described above, combined into a total information.
In an alternative manner, it is, in accordance with the invention, advantageous to measure the geometrical relation of the projection unit and of the image capture unit to one another through the fact that the pattern used is projected both onto the object as well as onto the image capture unit, and the pattern resulting on a defined surface rigidly connected with the image capture unit is recorded by one area of the image converter. In this sense, the image converter can record the object with one part of its areal field of view, and record the pattern projected onto the image capture unit with the other part.
In the evaluation of the image information, the geometrical position of the two units to one another is, in accordance with the invention, first of all advantageously computed from the distortion of the sample projected onto the surface of the image capture unit, which is, for example, flat and constructed as ground glass disk, by means of triangulation, and the object is, in addition, spatially measured, in relation to the assignment of both of the units to one another, in the way and manner as described above.
A process in which, in order to prevent multiplicities in the calculations of triangulation, coded patterns are projected onto the object, is already known from the German patent application xe2x80x9cA process for increasing the significance of the three-dimensional measurement of objectsxe2x80x9d of the date Sep. 12, 1996 by the same applicant.
It is, in accordance with the invention, particularly advantageous to use coded patterns for the determination of the coordination of the projection unit and of the image capture unit to one another in such a manner.
A process and a device in which beam paths can be effectively or ineffectively switched in an alternating manner by means of optical means placed synchronously with the alternating image current of the image converter are already known from the German patent application number 196 36 354.3 by the same applicant.
It is, in accordance with the invention, particularly advantageous to synchronously switch the direction of view of the image capture unit, such as by means of two surface-active LCD-unitsxe2x80x94which are positioned on two optical axes of a beam splitting devices, for examplexe2x80x94with the image alternating frequency of the image converter in such a manner that they are alternately optically permeable and non-permeable. This has the advantage that, for the purpose of determining the geometrical coordination of the projection unit with the image capture unit, the field of view of the image converter, and thereby the resolution in the representation of the object, does not have to be reduced.
It is, in accordance with the invention, advantageous to additionally provide further optical devices for the projection of a pattern which is different, for example, onto the image capture unit on the projection unit.
In an alternative manner to the surface rigidly connected with the image capture unit that has been described above, the pattern can, in accordance with the invention, be advantageously projected onto a separately-positioned surface with a known spatial extension, in order to determine the coordination of the image capture unit and of the projection unit with one another, and can be represented on the image converter of the image capture unit.
A second image capture unit for the purpose of determining the coordination of the image capture unit and of the projection unit with one another can advantageously be provided in accordance with the invention.
All of the configurations of the process and of the device that are described in this invention can, in a particularly advantageous manner, be combined with one another in multiple ways and manners in accordance with the invention.
The device for the three-dimensional measurement of objects of the type described above solves the task in accordance with the invention through the characteristics that:
a) At least one electronic surface image converter which uses an optical device suitable for the capture of images is used;
b) At least one device which carries out the digitalization of the output signals of the at least one image converter and makes these data available to a data processing apparatus is used;
c) At least one device which permits the projection of a pattern is used;
d) The devices for the projection of the pattern and for the capture of the image are constructed independently of one another;
e) At least one data processing apparatus which uses suitable circuits and/or algorithms is suited for the calculation of 3D coordinates in accordance with the triangulation process; and:
f) At least one device is suited for the determination of the geometrical coordination between the projection unit and the image capture unit.
In one configuration of the device in accordance with the invention which is particularly advantageous, the image capture unit is suitable for the capture of at least two successively following images.
One configuration of the device in accordance with the invention which is particularly advantageous provides devices which can store the data of at least two individual images.
One configuration of the device in accordance with the invention which is particularly advantageous provides devices which process the data of at least two individual images, and are able to combine these by means of suitable circuits and/or algorithms.